Using Dynamics 365 Remote Assist and HoloLens 2 for advanced communication in healthcare
Written by Md Shamiul Haque on 27.09.2022
Image: Hospital Clínic de Barcelona. Francisco Àvia. Creative Commons licence.
Augmented reality has numerous applications in a variety of industries. Healthcare or Telemedicine is no exception. Augmented reality (AR) based remote-assist solutions are one of the emerging applications in the healthcare industry. It can also be used for medical education and training.
Undoubtedly, HoloLens 2 is one of the strongest AR/MR (Mixed Reality) glasses amongst the currently available ones. Dynamics 365 Remote Assist is a mixed reality tool/software that enables remote collaboration between the HoloLens 2 and other Microsoft Teams desktop/mobile user(s). From the official Microsoft website, the following use cases of remote assist have been accumulated:
Collaborative maintenance and repair
Remote inspections
Knowledge sharing and training
Making a mixed reality video call is not so complicated, all it takes is the front-line caregivers to be on the HoloLens 2 and the specialist doctors to be on a desktop/mobile device with Microsoft Teams. Details on the usable devices can be found in this link. An overview of the annotations and other tools of the software can be found here.
Remote assistance can provide tons of benefits in the telemedicine sector. Not only does it solve problems in real time, but it can also provide mobility to the remote doctors without needing them to move. More effortless workflows are created by combining video, screenshots, and annotations. Drawings and arrows can be used by both on-site health care professionals and remote doctors to pinpoint specific parts of the patient's body in case of inspection or surgeries. Remote expert doctors can place reference images, schematics, and other useful information in the front-line health professional’s physical space, allowing them to refer to the schematic while working heads-up and hands-free on HoloLens. Dynamics 365 Remote Assist can also be navigated via voice commands to provide a hands-free experience.
One practical example of using the HoloLens in healthcare for remote assistance can be found here. The National Health Service of UK have strongly benefited by using HoloLens and Remote Assist during the COVID-19 pandemic. Doctors wearing the HoloLens 2 on COVID-19 wards could hold video conferences with colleagues and experts from all over the world. They were able to receive advice while also interacting with the caller and the patient. Medical notes and x-rays could also be displayed in the wearer's field of view alongside the call. Source: https://cloudblogs.microsoft.com/industry-blog/en-gb/health/2020/07/07/hololens-and-remote-assist-helps-the-nhs-provide-patient-care/
The Project med4PAN aims to use XR and 5G in chronic wound monitoring. Remote Assist and the HoloLens 2 could be one vital solution to this use case. At this moment, HoloLens 2 supports the following two methods for connecting to cellular and 5G networks:
An ad hoc Wi-Fi network provided by the cellular device, commonly referred to as a "Hotspot"
Limited support for USB-C tethered devices
A limited number of devices that present themselves as an ethernet adaptor can be used with Windows Holographic version 2004 and later. More details can be found in this link and here https://www.youtube.com/watch?v=52InDICzTcs&ab_channel=MicrosoftHoloLens
The augmented reality remote assistance solution is extremely beneficial to the concept of telemedicine or telehealth. It is fair to say that AR remote assistance has paved the way for the future of telemedicine. AR-based devices like HoloLens 2 can also provide more engaging and effective training for medical students. The coming years are crucial for the growth of this revolutionary technology and the healthcare sector as well. Med4PAN aims to contribute to the research associated with the growth of this above-mentioned technology and how 5G can be used in such collaborating scenarios.